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ABSTRACT 



A study was undertaken to determine (1) if six 
student teachers who received instruction (in eight weekly individual 
or group training and feedback sessions) in the. use of Bloom’s 
taxonomy would operate within the classroom at a higher cognitive 
level than a control group of six who received equal time instruction 
using a placebo-type treatment; (2) if two groups of trainer 
observers (administrators who were experienced in the taxonomy and 
cooperating teachers who were introduced to the taxonomy in much the 
same manner as their student teachers) could agree between and within 
themselves as to the cognitive level of questioning occurring within 
the classroom during the teacher-pupil dialogue as measured on the 
Teacher-Pupil Question Inventory '(TPQI) ; and (3) if there were a 
difference in the percentage of above-memory questions asked by 
teachers who had training in the use of the taxonomy and those who 
had no training. Each student teacher made a weekly tape recording of 
a social studies lesson; tapes of the third, fifth, and seventh weeks 
were rated by both groups of observers. .Analysis of the data, using 
chi square and analyses of variance, showed a significant difference 
favoring the experimental group of student teachers, and test results 
found no significant difference in the-rating scores of the 
observers, (Conclusions and recommendations are included. Appendixes 
contain the TPQI and a summary of the* taxonomy, as modified' by 
Sanders in 1966.) (JS) 
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INCREASING THE COGNITIVE LEVEL OP CIASSROOM QUESTIONS m 
SOCIAL STUDIES: AN APPLICATION OF BLOOM'S TAXONOMT 

George T. Farley 

Pittsfield, Massachusetts Public Schools 

and 

Ambrose A. Clegg, Jr. 

University of Washington 

An increasing body of evidence suggests that the majority of classroom 
questions, whether oral or written, are limited to those that require little 
more than a memorized response (Clegg, 1968) . Despite the high level 
objectives often found in social studies curricula, students are seldom 
asked to apply, analyze, synthesize, or evaluate the body of facts, concepts 
or generalizations they study. Since classroom questioning is an integral 
part of cognitive classroom behavior, it is essential that methods be found 
that will enable teachers to raise the cognitive level of their questions. 

Related Research 

In the literature reviewed by Farley (1968) a number of studies were 
cited which dealt with various aspects of questioning: teacher's oral 

questions in the primary grades (Floyd, i960); in secondary school social 
studies classes (Davis and Tinsley, I967), in textbooks (Davis and Hunkins, 
1966); and in questions relating to Guilford’s model of cognitive processes 
(Gallagher, I965) • Each of these studies pointed clearly to the predominence 
of teacher questions at what Bloom (1956) has called the knowledge level. 
Operationally defined, such questions require students to recall a response 
from memory. In a series of studies Hunkins (1967, 1968, 1969) has shown 
the differential effect upon learning of questions specifically prepared at 
the analysis and evaluation elvels of Bloom’s taxonomy. 



The present study took a different approach from those reported above. 
The investigators sought to determine if there would be an increase in the 
cognitive level of teacher’s classroom questions when training was provided 
in the use of Bloom's taxonomy. By making teachers aware of the different 
cognitive levels, it was hypothesized that there would be an increase in the 
frequency and percentage of higher level questions asked by teachers. This 
study makes use of a scoring instrument known as the Teacher Pupil Question 
Inventory (TPQI) developed by Davis and Tinsley (1967) > and certain 
procediires for the taining of teachers and observers in the use of the 
taxonomy which were developed in an earlier study by Clegg, Farley and 
Cvirran (1967). As on the Davis-Tinsley study, a modified version of Bloom's 
taxonomy was used (Sanders, 1966). Operational definitions and examples of 
each category are given in Appendix A. 

The present study differs from those mentioned above in two respects: 
(1) The scores of the experimental group were compared with those of a 
control group, and (2) a placebo-type treatment was used with the control 
group to lessen the possible influence of a Hawthorn effect since the treat- 
ment variable was being used with only one group and there was a possibility 
of interaction among subjects outside the experimental situation. 

Objectives 

This study was undertaken to deteimine if: 

1) Student-teachers who had received instruction in the use of Bloom’s 
taxonony would operate within the classroom at a higher cognitive level 
than student-teachers who had not received instruction in the taxonory. 

2) Two groups of trained observers (a group of raters, and a group 

of experienced classroom teachers) could agree between and within themselves 
as to the cognitive level of questioning occurring within the classroom 
during the teacher-pupil dialogue as measured on the Teacher-Pupil Question 
Inventory (TPQI). 
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3) There were a difference in "bhe percentage a DOve “memory quesuions 
asked by a group of student teachers that had had training in the use of 
Bloom’s taxonony and two other groups: (a) a comparable group that had 
had no such training, and (b) the group reported previously by Davis and 
Tinsley (1967) which also had had no training in the use of the taxonon^*’. 

Setting 

This study ^/?as conducted in two randomly selected communities in 
Western Massachusetts during an eight week student-teaching period in 1967- 
68. It involved twelve student teachers from the University of Massachusetts 
who were assigned to a participating elementary school in one of the two 
communities, thus making two groups of six student teachers each. The 
groups were then designated as experimental and control groups. There was 
no evidence of any systematic bias in the selection or assignment of the 
student teachers to the groups. It was also determined beforehand that 
none of the student teachers had been introduced previously to the Bloom 
taxonomy. 

Procedure 

During the eight week period, one of the experimenters met with the 
student teachers once a week at the two schools according to a prearranged 
schedule for individual or group training and feedback sessions . 

The schedule for the experimental group varied slightly from that of 
the control group since only the experimental group was to have knowledge 
of the taxonomy. Instead of discussing the taxonomy with the control 
group, equal time was devoted to discussing the Flanders system of inter- 
action analysis, which was used as the placebo-type treatment. 

Each student “teacher in the experimental and the control group was 
requested to make a weekly tape recording of a social studies lesson. Only 
the tapes of the third, fifth, and seventh weeks were used for evaluation 
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purposes relating to this experiment, although all the tapes were used by 
the experimenter in the feedback sessions mentioned above. 

The experimenter also met each week with the experienced teachers 
(called co-operating teachers) to whom the student-teachers of the 
experimental group had been assigned. The co-operating teachers were 
introduced to the taxonony in much the same manner as were the student- 
teachers. On the fourth, sixth, and eighth weeks of the experimental 
period, the co-operating teachers were asked to rate, by means of the TPQI 
a randomly selected portion of each tape made during the previous week. 
Only tapes from the experimental group were rated by the co-operating 
teachers . 

To avoid the confounding effect of experimenter bias, neither of the 
two writers of this paper participated in rating any of the tapes. A 
second group of raters was made up of two public school administrators^_ 
called investigators. These investigators were doctoral candidates in an 
educational research program and had extensive knowledge of the taxonoiny 
and the TPQI. The investigators listened to and rated the same taped 
segments as the co-operating teachers. The investigators also rated a 
like number of tapes associated with the control group. In recording the 
data on the succeeding pages the investigators are listed as while 

the co-operating teachers are listed as 

As was stated earlier, the measuring instrument was a Teacher Pupil 
Question Inventory (TPQI) . The TPQI is a single sheet which provides 
space for the rater to record the level of each question and the level of 
the response (see Appendix B). Twenty scores (ten teacher questions and 
ten pupil responses) were recorded from each lesson. The sum of these 
scores was used as a composite achievement score of one student -teacher 
as determined by one rater. 



Anothor phase of the experiment vas to tabulate the frequency st 
which each level of question was used. Percentage figures were then 
obtained and were used for testing purposes. Only the investigators* 
ratings (R^-Rg) were used in this part of the study. 

To determine if the objectives, stated earlier, had been achieved, 

four null hypotheses were tested. They were: 

1) There is no difference in the level of cognitive classroom 
behavior achieved by a group of student -teachers who have had instruction 
in the use of the Bloom-Sanders taxonomy, and a group of student -teachers 
who have had no instruction in the use of the taxonomy# 

2) There is no difference between investigators their 

rating of student -teachers , using the TPQI as a measuring device • 

3) There is no difference among investigators (R^-Rg) and co- 
operating teachers (R^-Rq) in their rating of student -teachers, using the 
TPQI as a measuring device. 

4) There is no difference in the percentage of memory questions 
asked by a group of student -teachers who have had training in the use of 
the taxonomy (experimental group) and (l) a group of student-teachers 
who have had no training in the use of the taxonomy (control group); and 
(2) a group of student -teachers previously reported in the literature who 
have had no training in the use of the taxonomy (Davis and Tinsley, 1967)* 

Design 

To test the first two hypotheses, a four factor design with repeated 
measures was used. The factors included (l) the treatment variable of 
whether the student-teachers had training in the use of the taxonomy, 

(2) the student -teacher variable, (3) the rater (investigators) variable, 
and (4) the trial variable. 
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To test the third hypothesis, a three factor design with repeated 
measures was used. The factors included (l) the student-teacher variable, 
( 2 ) the rater (two investigators and six co-operating teachers) variable, 
and ( 3 ) the trial variable. 

The data associated with the factors involved in hypotheses one, 
two, and three were examined by the analysis of variance method. 

A 2 X 2 chi-square table was used to obtain an answer to the fourth 
hypothesis . 

Findings 

The data used to tes+ hypotheses one and two are shovm in Table 1. 
This table is made up of the investigators* ratings of both the experi- 
mental and control groups. Co-operating teachers’ rating do not apply 
here. 



Table 1 



Investigators’ and Rg) Composite Scores for 
Experimental (T^) and Control (Tg) Groups 
Over Three Time Time Periods (C^-C^) 
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An snslysis of vsrisnco tss'fc wos usod. "to o"b"fc3in snswors p0r'tainxng 
■fco hypothGsos on© and. "bwo# RosuXts of "this "tesl; are shown in Tshie 2» 

Table 2 

SuMnary of Analysis of Variance Test 
of Mean Scores for Treatments (T) 
and. Investigators (r) 



Source 


Sum 

of 

Sq.uares 


Degrees 

of 

Freedom 


Mean 

Squares 


Observed 

F 

Values 




Treatments 


1208.68 


1 


1208.68 


11.88 


* 


Time Periods 


57.03 
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28.51 


.78 


NS 


Raters 
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10.13 


.53 


NS 


Student Teachers 
(within treatments) 


1017. 1^ 
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101.71 






Treatments, Time 
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1.27 
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Time.:-. Student 
Teachers 

(within treatments) 


731.11 
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Student Teachers 
- Raters 

(within treatments) 
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Treatment & Time 
Raters 
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46.93 


5.80 


NS 
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161.89 
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8.09 







Teachers Raters 
(within treatments) 
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Test results showed significant differences (p .01) "between treat- 
ments. Therefore, hypothesis one was rejected. It was concluded that 
student -teachers who have had training in the use of the taxonon^r 
achieved a higher level of cognitive classroom "behavior than student- 
teachers who have had no instruction in the taxonoirQr, 

The results also showed no difference between investigators. 
Therefore, hypothesis two was accepted. It was concluded that investi- 
gators, using the TPQI as a measuring device, agree in their ratings 
of student -teachers. 



The data necessary to test hypothesis three appear in Table 3* 
The table contains scores relating only to the experimental group. 
Both the investigators’ (R^-Rg) and co-operating teachers’ (R^-Rg) 
ratings are included here. 



Table 3 



Raters’ (R^-Rg) Composite Scores for Experimental Group 
(T^) Over Three Time Periods (C^-C^) 
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Hypothesis three was tested by means of analysis of variance. 

Results are shown in Table 4. 

Table 4 

Summary of Analysis of Variance Test 
of Mean Scores for Raters (R^-Rq) 



Source 


Sum 

of 

Squares 


Degrees 

of 

Freedom 


Mean 

Square 


Observed 

F 

Value 




Time 


408.85 


2 


204.42 


.95 


US 


Student Teachers 


4141.64 


5 


828.33 


15 -IT 


-K- 


Raters 


467.44 


T 


66.78 


1.22 


W3 


Time Student 
Teachers 


2142.74 


10 


214.27 


5.14 


* 


Time Raters 


1068.60 


14 


76.33 


1.83 


RS 


Student Teachers 
Raters 


1910.80 


35 


54.59 






Time -r Student 
Teachers x Raters 


2915-15 


70 


41.65 







*Significant (p .01) 

The results showed no significant difference among raters. 
Therefore, hypothesis three was accepted. It was concluded that there 
is no difference among investigators and co-operating teachers in their 
ratings of student teachers, using the TPQI as a measuring device. 

It is noteworthy to point out that tests relating to student- 
teachers showed significant differences. Although it was not the 
intention of this study to prove individual differences among teachers, 
the statistical design made such a test possible. Of course, the 
finding was ,, expected . 



To obtain answers pertaining to hypothesis four, it was first 
necessary to devise frequency and percentage charts for both the 
experimental and control groups. Subsequently, percentages of memory 
and above-memory questions, asked by each group, were obtained* The 
frequency and percentage tables appear in Table 5* Only the two 
investigators rated teachers questions in this phase of the 

study. 

Table 5 

Frequency and Percentage of Student -Teachers 
Questions Arranged by Cognitive Levels 
as Rated by Investigators (R^-R^) 

Control Group 



Cognitive 

Levels 


1 

1 


f ^ 


®3’ 

f i> 


i 

f 




" 5 ’ 

f s6 


f 

f 


>6' 

f 


Total 
f ^ 


1 Memory 


18 


30 


23 


38 


18 


30 


25 


42 


11 


18 


18 


30 


113 


31 


2 Translation 


29 


1^8 


29 
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2k 


40 
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58 
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51 


3 Interpretation 
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13 


10 


IT 
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14 


23 


T 


12 


60 


IT 


h Application 
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0 


0 


0 


0 


0 


5 
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5 Analysis 
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0 


0 
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0 


0 
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6 Synthesis 
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0 


0 


0 


0 


T Evaluation 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 



Experimental Group 



Cognitive 

Levels 


S-, 

f 


Sp 

f 


f 




S. 

f 


f H 


Total 

f 


1 


Memory 


5 


8 15 25 


9 15 


23 38 4 


.7 


2 


3 


58 


16 


2 


Translation 


29 48 29 48 19 32 


26 43 14 
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35 


58'' 


152 


42 
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Interpretation 25 


42 15 25 30 50 


11 18 
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Percentages of memory questions and above-memory questions vere then 
used to contrast, by chi-square, the experimental group with (l) the 
control group; and (2) the Davis-Tinsley group. Figures for the Davis- 
Tiiisley group were 6l per cent memory and 39 psr cent above-memory. 

Test results related to Hypothesis Four are given in Table 6. 

Table 6 



Chi-Square Tests Comparing Experimental 
Group With (l) Control Group And 
(2) Davis-Tinsley Results 



Control Group 

Experimental 

Group 


Memory 

Level 

31 

16 


Above 

Memory 

69 

84 


Chi- 

Square 

6.3 * 




Memory 


Above 


Chi- 




Level 


Memory 


Square 


Davis-Tinsley 


61 


39 


. 42.6 ** 


Experimental 

Group 


16 


84 





* Significant (p. < .05) 
** Significant (p. < .01) 
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Since significant differences were found in both the above tests, 
Hypothesis Four was rejected. Therefore, it was concluded that there 
was a difference in the percentage of memory questions asked by a group 
of student“teachers who have had training in the use of the Taxonomy and 
and two groups of student-teachers who have had no training in the use 
of the Taxonomy . 

Conclusions 

A number of conclusions can be drawn from this study. First, in- 
struction in the use of the Taxonomy as a tool for clearly identifying 
the different levels of intellectual behavior does make difference in 
the quality (cognitive level) of questions that teachers ask. Second, 
since the TPQI is simple to use and appears to have high reliability, it 
nay be a useful device for helping teachers to analyze and evaluate their 
own performance. Third, the close agreement among raters corresponds 
with earlier findings reported by Clegg et al ( 196 ?) and bolsters 
Clegg *s suggestion that the Taxonony can serve as a common educational 
language for the improvement of instruction. 

Finally, it must be conceded that social study goals which call for 
higher levels of thinking are not being achieved through the use of 
questioning. A careful look at the frequency and percentage tables for 
both groups (Table 5 ) clearly shows that seldom was a question asked 
which went beyond the interpretation level. Consequently, the findings 
reported here provide additional support to those of Gallagher (1965), 
and Davis and Tinsley (196T), discussed earlier. Together, they 
challenge the time-honored practice of asking questions as a method of 
teaching. A corollary to the above is that teachers are not being given 
proper training in questioning techniques to successfully implement the 

higher levels of thinking in their classrooms. 
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Recommendations 



Th6 findings reported herein appear to suggest a promising approach 
toward the improvement of teacher education and pupil achievement* It 
appears desirable to include study of the Taxonomy in the teacher- 
training curriculum as a tool for raising the level of classroom learn- 
ing. This might he done: (l) combining the teaching of the Taxonomy 

with some approach to the study of cognitive tasks; such as Taha’s (l9oT) 
schema; or (2) using the Taxonony with some adaptation* of micro- 
teaching. 

The success of the TFQI as a measurement of cognitive behavior 
suggests that the Taxoncmy could he used with other measuring instru- 
ments. The Taxonomy might be used to add a cognitive dimension to the 
OScAR, Medley and Mitz$l-(l958, 1963), or it might he employed with 
Flanders’ In raction Analysis to determine the effect of teaching style 
on cognitive achievement. 

In view of the small size of the sample used, this study should he 
renlicated on a larger scale to verify the findings and conclusions 
presented here. A suggested modification might he to teach the 
Taxonomy in three phases. Phase one would concentrate on encouraging 
teachers to ask above -memory questions. Phase two would emphasize 
convergent thinking questions (applications, and analysis), while phase 
three would emphasize divergent thinking questions (synthesis, evalu- 
ation) . 

Several other questions might also he tested in subsequent studies. 
For example: (l) Will the cognitive response level increase if the 

pupil is taught to become aware of the level of his expected response? 
(2) Does eliciting longer answers from the pupil tend to raise the 
cognitive level of the response? 
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(3) Is there a correlation between the type of teacher (according to 
the Flanders model) and the cognitive level attained in the classroom 
(as measured by the Taxonomy ) ? (^) Do students operate at the same 

cognitive level in one subject as they do in another? 

, ^ Summary 

This study sought to determine if there would be an increase in the 
cognitive level of teacher classroom questions when training was pro- 
viced in the use of Bloom’s taxonomy. By making teachers aware of the 
different cognitive levels, it was hypothesized that there would be an 
increase in the frequency and percentage of higher level questions asked 
by teachers. Tvro additions^, questions were askedi (l) Could a group 
of trained observers agree among themselves as to the cogniti/e level 
of the questions asked? (2) Was there a difference in the percentage 
of ’’above-memory” questions asked by teachers who had training in the 
use of the taxonomy and those who had no training. 

Analysis of the data showed a significant difference favoring the 
experimental group of student-teachers. It was concluded that teachers 
who employ the taxonomy as a teaching tool will ask higher level ques- 
tions in the classroom. Test results also found no significant differ- 
ence in the rating scores of the ohser\^ers when employing the TPQI. It 
was suggested therefore, that the simplicity and the reliability of the 
TPQI make it a useful device for helping teachers to analyze and evalu- 
ate their own performanc3, and that the taxonomy can serve as a common 
educational language for the improvement of instruction. It was recom- 
mended that study of the taxonomy be included in the teacher-education 
curriculum as a tool for raising the level of classroom learning, and 
that the taxonomy be added to as a cognitive dimension to existing in- 
struments for observing classroom behavior such as the Medley and Mitzel 

OScAR or the Flander’s system of interaction analysis. 
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APPENDIX A 



A S» 1 ARY OF 

BLOOM’S TAXOWOl^y OF EDUCATIONAL OBJECTIVES: 

COGNITIVE DOMAIN AS MODIFIED BY SANDERS (I966) 

Meinory - ability to recogniz© 02* rGcall infonnatioii presented earlier. 

It includes definitions, generalizations, and values. 

Translation - ability to express information in a different form. A 
student translates when he expresses something in his own words. He 
also translates when he explains what he sees in a picture or draws a 
picture of something that he heard or read* Acting out of historical 
events is another form of translation. 

Interpretation - ability to perceive a relationship between two ideas. 
This relationship is perceived by the student through use of his own 
common sense. 

Questions which may be placed at the interpretation level tell the 

student explicitly what to do and have one or only a few logical answers. 

(e.g. Compare the Northern and Southern view on Slavery prior to i 860 .) 

The different kinds of relationships include comparison, implication 

(informal deduction), generalization, (infonnal induction), value, 

skill of definition, quantity, and cause and effect. 

Note : If, in making comparisons, the student employs a more 

formal approach, using the rules of logic, he is operating 
at the analysis level. 

Application - ability to use previously learned Ideas, principles, and 
generalizations in new situations. This category is designed to give 
students practice in the transfer of training. 

Questions are considered to be at the application level if they 
have problem-solving power, deal with the whole of ideas and skills ^ and 
include a minimum of directions as the' student is expected to know what 
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APPENDIX A (continued) 

to do. (e.g. In i860, how did the North and South differ?) This ques- 
tion is much broader than the example used in the interpretation category. 
The difference between interpretation and application questions is that 
in the former the student is asked to illustrate the use of some abstrac-* 
tion to appropriate use. 

Analysis - ability to break dov/n material into its parts by employing 

the formal rules of logic. Since very little teaching time is given to 

formal instruction in the parts and processes of reasoning (induction, 

fallacies, deduction, and semantics), very few analysis questions can be 

employed in the average classroom. 

I^ote: Sanders does suggest that analysis questions 

related to developing generalizations by the 
inductive approach may -be used with elemental^ 
school children. 

Synthesis - ability to draw elements from many sources to form a unified 

structure not clearly perceived before. 

Two important characteristics of synthesis questions are that they 
allow the student great freedom in seeking a solution, and that their 
solution requires a product in the form of a unique plan, a communica- 
tion, or a set of abstract relations. 

Evaluation - ability to raalce a judgement about the value of ideas, 
solutions, methods, materials, etc., using criteria developed by the 

student, himself (not by the teacher) • 

Evaluation questions require the student to perform two operations. 
First, he must establish appropriate standards or values. Second, he 
must determine how closely the idea or object meets these standards. 

It is important to remember that evaluation questions deal with 
values and not with facts or opinions. They are always somewhat 
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APPENDIX A (CONOTUED) 

subjective because either the standard cannot he proved to be correct, or 

the idea to he judged cannot he proved to violate the standard. 

Note: Sanders holds that questions in which the teacher 

specifies the values for wahing a judgement fall 
under the interpretation category. 
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